Sequential labeling and classification of data (also referred to as sequence tagging herein) has many applications, including those in natural language processing and speech processing. Some example applications include search query tagging, advertisement segmentation, and language identification/verification. Several different machine learning techniques have been applied to sequence tagging problems, such as conditional random fields (CRFs) and neural networks.
Conditional random fields (CRFs) are discriminative models that directly estimate the probabilities of a state sequence conditioned on a whole observation sequence and are also known as information extraction tasks. For example, frames of audio signal data may be converted to features, with the state sequence predicted on all the frames. Because CRFs can be utilized for numerous different tasks and because they can achieve high accuracy with minimal tuning, conditional random fields are the most widely used machine learning technique applied to sequence tagging problems. However, CRFs fail to take advantage of unlabeled data.
It is with respect to these and other general considerations that embodiments disclosed herein have been made. Also, although relatively specific problems may be discussed, it should be understood that the embodiments should not be limited to solving the specific problems identified in the background or elsewhere in this disclosure.